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PREFACE 


This  work  was  conducted  under  Project  2313,  Task  2313T3,  Work  Unit 
2313T312,  Coqnitive  Aspects  of  Fliqht  Training.  The  purpose  of  this 
basic  research  was  to  develop  a  methodoloqy  for  investiqatinq  the 
decision-making  processes  involved  in  pilotinq  an  aircraft.  The 
author  is  grateful  to  1st  Lt  John  Brunderman,  who  accomplished  the 
developmental  proqramninq  for  the  computer-based  vehicle  control  task 
used  in  the  research. 


t 


Velocity  Control  Dec  is  ion -Making  Ability: 

Relationship  to  Flying  Capability  and  Experience 

I.  INTRODUCTION 

A  number  of  vehicle  control  tasks  involve  maintaining  some  predetermined 
steady  state.  Driving  an  automobile  on  a  straight  road,  steering  a  ship  on  a 
constant  heading,  and  flying  an  aircraft  at  constant  heading  and  climb  rate 
are  examples  of  such  tasks.  In  these  types  of  tasks  the  operator's  function 
is  primarily  one  of  detecting  and  correcting  error,  in  the  form  of  deviations 
from  the  desired  motion  path.  For  the  most  part,  the  operator  acts  as  a 
simple  closed-loop  controller  whose  performance  is  determined  primarily  by  its 
sensitivity  to  error  and  by  its  control  lag  (Jagacinski,  1977). 

In  a  number  of  other  tasks,  the  operator's  role  is  much  more  complex.  The 
operator  seeks  not  simply  to  maintain  a  steady  state  but  rather  to  achieve  a 
particular  vehicle  state  through  execution  of  a  sequential  process  of  velocity 
changes.  In  these  tasks  the  operator's  actions  are  directed  toward  the  goal 
state,  which  differs  from  the  current  state  and  from  intermediate  states. 
Intermediate  vehicle  states  are  important  only  insofar  as  they  affect  the 
ultimate  vehicle  state.  Task  sequences  can  be  relatively  short  and 
uncomplicated,  as  in  parallel  parking  a  car,  or  quite  long  and  complicated,  as 
in  the  case  of  driving  home  in  rush  hour  traffic  in  time  to  see  the  news  on 
TV.  In  these  instances  control  performance  is  determined  both  by  the 
operator's  interface  with  the  vehicle  and  by  the  operator's  performance  as  an 
information  processor  and  decision  maker. 

Describing  performance  on  these  task  sequences  is  complicated  by  the  large 
number  of  control  actions  involved  and  by  the  fact  that  the  optimum  action  at 
a  given  point  in  the  sequence  is  determined  by  earlier  events  both  internal  to 
the  operator -vehicle  system  and  external  to  it.  For  example,  the  decision  on 
whether  to  turn  right  at  an  intersection  or  to  proceed  through  can  depend  on 
whether  the  vehicle  will  reach  the  intersection  while  the  signal  is  green 
(internal),  how  long  the  red  will  last,  how  well  traffic  is  moving  on  the  two 
streets,  and  the  length  of  the  alternate  paths  (external).  The  operator 
steers  the  vehicle  according  to  a  goal-directed  scenario  which  is  updated 
continuously  in  response  to  changing  conditions.  The  correctness  of  any 
particular  control  action  in  the  sequence  is  determined  by  two  factors:  the 
correctness  of  the  scenario  and  the  appropriateness  of  the  control  action  to 
the  scenario. 

At  a  macro  level  vehicular  control  scenarios  are  very  similar  to  scripts. 

A  script  is  a  set  of  related  facts  which  are  organized  hierarchically  and 
sequentially  (Smith,  Adams  &  Schorr,  1978).  In  the  motion  scenario  facts 
might  be  events  or  control  actions  which  are  organized  by  the  importance  or 
probability  of  occurrences  at  particular  points  in  the  process.  Schvaneveldt 
et  al  (1982)  have  shown  experienced  fighter  pilots  to  organize  major  events 
and  actions  related  particular  maneuvers  in  this  way.  Facts  in  these  scripts 
are  aircraft  states,  pilot  actions,  effects  of  these  actions,  controls  and 
instruments,  and  actions  of  other  combatants.  These  facts  are  organized 
primarily  along  a  dimension  of  temporal  criticality,  analogous  to  a  script, 
and  secondarily  along  a  dimension  of  relative  energy. 

At  a  finer  level  of  detail,  involving  individual  control  inputs,  vehicle 
control  can  be  thought  of  as  a  sequential  process  of  goal  directed  decisions 
and  control  actions.  While  the  second-by-second  control  process  may  involve  a 
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sequence  of  control  decisions,  it  is  less  well-suited  to  a  scripts  type  of 
analysis  since  events  occur  rapidly  and  decisions  are  often  based  on 
information  which  is  not  organized  semantically.  The  control  decision  process 
is  similar  to  closed-loop  tracking,  but  unlike  tracking,  it  consists  of  a 
sequence  of  actions  directed  at  establishing  a  particular  state  rather  than  at 
simply  maintaining  a  particular  state  with  minimal  error. 

In  the  goal-directed  control  process,  several  different  control  actions 
might  be  effective  in  achieving  the  desired  state.  The  operator's  decision 
about  which  actions  to  employ  may  depend  on  many  factors,  such  as  the  degree 
and  direction  of  motion  required,  the  probability  of  success,  and  the  cost  of 
failure  associated  with  a  particular  control  action  sequence.  In  the 
real-time  motion  control  task,  it  is  difficult  to  isolate  the  micro-decision 
making  process  from  perceptual  and  psychomotor  factors.  If,  for  instance,  a 
pilot  ends  up  with  excessive  airspeed  on  final  approach,  it  is  not  possible  to 
know  whether  the  error  arose  from  the  pilot's  failure  to  perceive  the  airspeed 
buildup  or  from  an  inability  to  make  the  appropriate  control  inputs  or  whether 
the  pilot  saw  what  was  happening  and  decided  that  everything  was  just  fine. 

One  way  of  isolating  the  micro-decision  making  process  is  to  present 
subjects  a  discrete-time  vehicle  control  task.  In  this  task  the  subject 
"steers"  a  computerized  vehicle  through  a  course  in  a  sequence  of  moves.  On 
each  move,  a  set  of  driving  equations  determine  the  motion  envelop  according 
to  the  vehicle  state  at  the  end  of  the  preceding  move  through  .  The  task  is 
self-paced,  and  the  subject  types  the  desired  move  on  a  keyboard.  De  Maio 
(1981)  compared  the  performance  of  Air  Force  pilots  with  that  of  non-pilots  on 
a  discrete-time  task  in  which  subjects  controlled  vehicular  movement  by 
adjusting  the  length  of  two  perpendicular  driving  vectors.  As  indexed  by  the 
number  of  moves  required  to  complete  the  course,  the  pilot  group  performed 
better  than  did  the  non-pilot  group.  The  pilots  also  made  longer  moves  on  the 
average,  although  the  average  total  track  length  was  the  same  for  both  groups. 

In  the  present  research,  a  discrete-time  vehicle  control  task  was  used  in 
which  the  driving  equations  more  closely  approximate  those  of  an  actual 
vehicle.  The  vehicle  performed  as  a  generic  aircraft  constrained  to  level 
flight.  The  work  examined  the  relationship  between  performance  on  the 
discrete-time  control  task  and  flying  capability  and  experience.  Performance 
on  individual  moves  was  examined  to  determine  the  decision  making  process 
leading  to  superior  task  performance. 


II.  METHOD 


Subjects 


Three  groups  of  Fighter-Attack-Reconnainance  (FAR)  qualified  pilots  served 
as  subjects.  These  included:  25  F-16  T-course  pilots,  nine  F- 16  B-course 
pilots  and  16  A-10  B-course  pilots.  The  T-course  pilots  had  700  to  4500  hours 
miltary  flying  time;  the  B-course  pilots  had  about  250  hours.  Three  groups  of 
students  in  Undergraduate  Pilot  Training  (UPT)  served  as  subjects.  These 
included:  eight  students  awaiting  the  start  of  training,  six  who  had  just 
completed  the  T-37  phase  and  10  who  had  just  completed  the  T-38  formation 
phase.  Average  flying  time  for  these  groups  was  roughly  4  hours,  70  hours  and 
130  hours  respectively. 
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The  Flight  Decision-Making  Assessment  Task  (FDAT)  is  a  discrete-time 
vehicle  control  task  programmed  on  a  Terak  8510/A  micro-computer.  The  FDAT 
driving  functions  are  generic  flight  equations  for  an  aircraft  constrained  to 
level  flight  and  scaled  to  fit  on  the  320  x  240  Terak  graphics  screen.  Three 
functions  determine  vehicle  performance  :  maximum  heading  change,  induced 
drag  velocity  penalty,  minimum/maximum  velocity  change. 

The  FDAT  involves  "steering"  a  vehicle  through  a  winding  course  in  a 
series  of  moves.  The  subject's  task  is  to  traverse  the  course  in  as  few  moves 
as  possible.  The  subject  makes  a  move  by  entering  two  numbers.  The  first 
number  entered  is  the  amount  of  heading  change  desired,  in  degrees.  Following 
this  input  the  vehicle’s  velocity  is  decremented  in  proportion  to  the  amount 
of  heading  change  by  an  "induced  drag"  function.  A  thrust/drag  function  then 
determines  the  amount  the  subject  may  increase  or  decrease  speed.  The  subject 
completes  the  move  by  entering  the  speed  desired;  that  is,  the  length  of  the 
move.  The  ground  track  is  then  updated  to  show  the  move.  The  new  heading, 
speed  and  associated  maximum  heading  change  and  the  induced  drag  penalty  are 
shown  on  a  digital  display  to  the  left  of  the  course. 

A  complete  listing  of  the  vehicle  performance  limits  is  given  in  Table  1. 
The  "stall"  or  minimum  velocity  is  7  units,  and  the  maximum  velocity  is  27 
units.  Velocity  units  are  pixels.  The  maximum  allowable  heading  change  reach 
minima  of  13°  at  stall  and  23°  at  maximum  velocity  and  a  maximum  of  48° 
at  the  14  unit  corner  velocity.  The  course  lengths  are  roughly  680  pixels. 

Table  1 

Vehicle  performance  limits  at  each  velocity 


Veloc itj 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 
27 


Heading 
Change 
Degrees 
Max imum 


Induced 
Drag 
Penalty 
Max imum 
0 
-1 
-2 
-3 
-4 
-5 
-6 
-7 
-7 
-8 
-8 
-8 
-8 
-9 
-9 
-9 
-9 
-10 
-10 
-10 
-10 


Velocity  Change 
Minimum  Maximum 


r  — jy  i  i  m  Mir  ti  ■ 


Procedure 


The  subject  sat  before  the  computer  terminal  with  the  experimenter  seated 
at  the  left.  The  experimenter  explained  that  the  purpose  of  the  research  was 
to  examine  the  decision  making  process  associated  with  aircraft  control.  The 
experimenter  then  made  a  series  of  demonstration  moves  showing  turning, 
accelerating,  and  the  effect  of  trying  to  exit  the  course  boundaries. 

Some  of  the  demonstration  moves  are  shown  in  Figure  1.  In  Figure  la  the 
initial  condition  is  shown.  The  initial  condition  has  heading  180°  and  a 
speed  of  13  units.  Figure  lb  shows  the  state  following  input  of  a  left  30° 
heading  change.  Speed  has  been  decremented  by  six  units  to  seven  units.  At 
seven  units  the  allowable  speed  change  (delta-v)  is  zero  unit  decrement  to  one 
unit  increment.  Figure  lc  shows  the  state  at  the  start  of  the  second  move. 

Heading  is  150°,  speed  is  eight  units,  and  the  maximum  heading  change  is 

19°.  A  line  eight  pixels  long  in  the  upper  left  represents  the  ground 

track.  Figure  Id  shows  the  third  move  just  before  input  of  the  speed 

command.  A  10-unit  move  along  heading  150°  was  made  on  move  two,  and  no 
heading  change  was  made  on  move  three  so  that  the  current  speed  is  10  units 

and  the  delta-v  range  is  -1  to  +4  units  for  a  move  length  range  of  nine  to  14 

units.  Figure  le  shows  the  state  following  move  three. 

Following  the  demonstration  moves,  the  subject  was  shown  the  effect  of 
trying  to  exit  the  course  boundaries,  which  was  to  reset  the  subject  at  the 
beginning  of  the  move  with  the  new  heading  and  a  speed  of  10  units.  The 

subject  was  also  shown  how  to  remove  input  mistakes  and  how  to  take  a  move 

over.  The  subject  then  completed  two  courses,  always  performing  the  easier 
course  first.  The  experimenter  was  present  throughout  the  session.  The 
subject  was  allowed  to  ask  questions,  but  questions  about  strategies  were  not 
answered.  _ _ _ _ 


(a)  Initial  condition 


(e)  State  after  completion  of  third  move 


Figure  1.  Examples  of  demonstration  moves  given  to  subjects 
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III.  RESULTS  AND  DISCUSSION 


Overall  Performance 

Overall  FDAT  performance  was  indexed  by  the  nunfcer  of  moves  required  by 
each  subject  to  traverse  each  course.  The  number  of  moves  ranged  from  38  to 
75  for  Course  1  and  from  40  to  37  for  Course  2.  The  mean  number  of  moves 
required  by  each  group  to  complete  the  two  courses  is  shown  in  Figure  2. 


Figure  2.  FDAT  performance 

An  analysis  of  variance  was  performed  on  the  combined  data  for  both 
courses.  The  results  of  this  analysis  are  shown  in  Table  2.  A  post-hoc 
analysis  of  three  selected  contrasts  was  performed  by  means  of  a  Dunn  test 
(Keppel,  1973). 

Table  2 

Analysis  of  Variance  for  Total  Number  of 
Moves  to  Complete  Two  Courses  by  Groups 


SOURCE 

SQUARE  D.F. 

F-RATIO  p 

TOTAL 

202.2 

73 

GROUPS 

839.3 

5 

5.4  .05 

ERROR 

155.3 

G8 

This  analysis  revealed  a  significant  difference  between  the  FAR  pilot  groups 
and  the  UPT  groups  (T{68)=  4.19,  jg  <  .05).  This  result  is  understandable 
since  the  FAR  pilots  have  both  more  experience  than  do  the  UPT  students  and 
more  capability  than  does  the  average  Air  Force  pilot.  The  FAR  pilots  ranged 
in  experience  from  250  to  4500  hours  of  military  flying  time  versus  0  to  130 
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hours  for  the  UPT  students.  In  addition  the  FAR  groups  represent  the  top  20 % 
to  40%  of  UPT  graduates  while  the  UPT  groups  represent  a  random  sample  of  all 
UPT  students. 

The  effect  of  flying  experience  alone  was  determined  by  looking  at 
differences  within  the  FAR  pilot  and  UPT  groups.  No  differences  were  found  as 
a  function  of  experience  either  within  the  FAR  groups  (T( 68)  <  1,  >  .05  , 

Figure  3)  or  within  the  UPT  groups  (Figure  2).  These  results  suggest  that  it 
is  flying  capability  rather  than  flying  experience  which  is  related  to  FDAT 
performance. 


B COURSE  T  COURSE 


Figure  3.  FDAT  performance,  T-course  vs  B-course 

One  surprising  difference  did  occur  within  the  FAR  groups.  The  F-16  B 
course  had  much  better  performance  than  did  the  A-1Q  B  course  ( T( 68)  =  2.73, 
p  .05).  This  difference  reflects  a  difference  in  flying  capability.  At  the 
"time  the  data  were  collected,  all  UPT  graduates  assigned  to  the  F-16  were 
Distinguished  Graduates  (DG);  that  is,  they  had  graduated  in  the  top  10%  of 
their  class.  On  the  other  hand,  fewer  than  10%  of  graduates  assigned  to  the 
A- 10  were  DG.  It  was  not  possible  to  ascertain  the  class  standing  of  some 
students,  who  had  participated  early  in  the  study,  but  an  analysis  of  variance 
was  performed  on  the  data  for  those  students  whose  class  standing  was  known. 
The  DG  pilots  evidenced  significantly  better  FDAT  performance  than  did  the 
non-DG  pilots  ( F(  1 ,19)  =  8. 3 ,  £  <  .05  ,  Figure  4). 

Internal  Performance  Analysis 

Task  requirements  and  the  vehicle's  handling  qualities  placed  a  high 
premium  on  maintaining  a  speed  equal  to  or  greater  than  14  units.  Due  to  the 
vehicle's  low  thrust/drag  ratio  it  was  not  possible  to  sustain  speed  during 
intense  cornering.  As  a  result  the  subject  needed  to  position  The  vehicle  on 
the  course  in  such  a  manner  as  to  permit  acceleration  and  to  time  turns  to 
minimize  heading  changes  and  resultant  loss  of  speed.  Speed  control  and 
timing  errors  were  identifiable  by  visual  inspection  of  individual  moves  and 
move  sequences. 


Figure  5  contains  examples  of  good  and  poor  performances  on  both  courses. 
Errors  frequently  consist  of  allowing  speed  to  bleed  off  until  the  vehicle 
"stagnates"  —  that  is,  loses  cornering  and  accelerative  power,  of  initiating 
turns  too  late,  or  of  making  unnecessary  heading  changes. 


Figure  4.  FDAf  performance,  DG  vs  non-DG 


(a)  Good  performance  on  Course  1 
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(b)  Poor  performance  on  Course  1 


(c)  Good  performance  on  Course  i? 
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(d)  Poor  performance  on  Course  2 
Figure  b  Examples  of  groundtracks 


In  order  to  quantify  speed  control 
of  moves  were  defined: 

1 .  Velocity  control 

a.  Extending  moves  - 

b.  Stagnating  moves  - 
2 •  Turning  control 

a.  Sustained  turn  moves  - 

b.  Positioning  moves  - 


and  timing  performance  four  categories 


5°  turn  and  accelerating 

deceleration  >_  2  units 
or  speed  <»  10  units 


turn  5°-30°  in 

constant  direction 

with  roughly  constant  speed 

turn  50-3QO 

for  only  one  move 

£  5°  turn  with  0-1  unit 

deceleration 


Velocity  control  performance  was  given  by  the  ratio  of  the  number  of 
extending  moves  to  the  number  of  stagnating  moves.  A  measure  of  turning 
control  performance  was  the  ratio  of  the  number  of  sustained  turn  moves  to  the 
number  of  positioning  moves.  A  measure  of  timing  was  the  total  amount  of 
heading  change  regardless  of  direction,  since  mis-timing  turns  necessitated 
corrective  maneuvers  which  increased  the  total  heading  change  for  the  course. 

In  order  to  determine  the  contribution  of  speed  control  performance, 
turning  control  performance  and  timing  performance  to  overall  performance  a 
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multiple  regression  of  the  two  ratios  and  the  total  heading  change  was 
performed  on  number  of  moves  for  Course  1  and  for  Course  2  separately. 

For  Course  1  all  three  internal  performance  measures  contributed 
significantly  to  overall  performance  (Table  3).  The  primary  driver  of 
performance  was  speed  control.  Timing,  as  measured  by  total  heading  change, 
also  contributed  substantially  to  overall  performance.  The  ratio  of  sustained 
turn  to  positioning  moves  measures  the  smoothness  of  heading  control  and  made 
only  a  small  contribution  to  overall  performance. 

Table  3 


Multiple  regression  analysis 
meaures  on  overall 

internal  performance 
performance 

Factor 

Course  1 
Beta-weight 

Course  2 
Beta-weiqht 

Extending /Stagnating 

-.46  * 

-.93  * 

Sustained/Positioning 

-.27  * 

NS 

Total  Heading  Change 

.38  * 

NS 

R 

.89  * 

.93  * 

*  p  <  .001 

The  results  of  the  regression  analysis  for  Course  2  differed  substantially 
from  those  for  Course  1  (Table  3).  Speed  control  performance  accounted  for 
over  85%  of  the  variance  in  overall  performance.  Neither  of  the  turn  control 
measures  contributed  significantly  to  overall  performance.  The  differing 
results  for  the  two  courses  probably  stems  from  a  design  difference  in  Course 
2.  Course  2  contained  3  turns  (out  of  6)  in  which  it  was  nearly  impossible  to 
avoid  complete  speed  stagnation.  As  a  result  speed  control  was  more  highly 
correlated  with  both  overall  performance  and  with  turn  control  performance  on 
Course  2  than  on  Course  1  (Table  4).  In  addition  a  much  greater  proportion  of 
moves  was  related  to  speed  control  on  Course  2  than  on  Course  1  (Table  5). 

Table  4 


Individual  correlations  between  internal  performance 
variables  and  overall  performance.  Course  2  shown  in  () 

Extending  Sustained  Total  Heading 
_ Overall  Stagnating  Positioning _ Change 


Overall 


Extending/Stagnating  79( 93) 
Sustained/Positioning  -,55( - .57 )  .39(.46) 


Total  Heading  Change 


,72( .69) _ -.59(-.77)  -.24(-.41) 
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Substantial  differences  were  found  between  pilot  groups  in  FDAT  control 
performance.  In  particular  FAR  pilots  showed  performance  which  was  superior 
to  that  o:  Undergraduates  Pilot  Trainees  on  the  whole.  Also  Distinguished  UP  I 
graduates  performed  better  than  FAR  pilots  who  were  not  DG.  These  difference 
appear  to  be  related  to  flying  capability.  Performance  on  the  FDAT  does  not 
appear  to  be  affected  by  flying  experience  per  se,  since  differences  in 
performance  as  a  function  of  flying  experience  were  found  neither  within  the 
FAR  groups  nor  within  the  UPT  groups. 

'  Table  5 

Proportion  of  moves  in  each  performance 
category,  SD  in  () 

Sustained  Posi-  Ext-  Stag- 
_ Turn _ tioninq  ending  natinq 

Course  1  .22(.17)  . 14( . 10)  , 32 ( .11)  .33(.20) 

_ Course  2  .2b(,26)  .03(.Q2)  .21(.08)  .50( .19) _ 

The  present  work  is  based  on  the  idea  that  the  process  of  steering  a 
vehicle  from  an  initial  energy-position  state  to  a  goal  energy-position  state 
can  be  described  as  a  sequence  of  vehicle  control  decisions.  Success  in 
achieving  the  desired  state  will  be  a  function  of  the  quality  of  the  control 
decisions  made  during  the  process  sequence.  The  FDAT  used  in  this  work  is  a 
discrete-time  vehicle  control  task  in  which  subjects  are  required  to  make  a 
sequence  of  decisions  regarding  three  aspects  of  vehicle  control:  speed 
control,  turning  control,  and  timing  of  turns.  Examining  internal  performance 
indicators  which  tap  these  decision  making  functions,  makes  it  possible  to 
quantify  these  decision  functions  and  also  to  assess  the  relative  importance 
of  the  decision  making  processes  to  overall  task  performance.  The  present 
results  show  that  overall  performance  can  be  described  by  a  linear  combination 
of  speed  control,  turning  control,  and  turn  timing  performance. 

When  the  FDAT  course  was  appropriately  designed,  as  in  Course  1,  the 
contribution  of  speed  control  in  the  regression  accounted  for  37%  of  the 
variance  in  overall  performance.  The  contribution  of  turn  timing,  as  indexed 
by  total  turn,  accounted  for  27%  of  the  variance  in  overall  performance. 
Turning  control,  as  indexed  by  the  ratio  of  sustained  turn  to  positioning 
moves,  had  the  smallest  contribution  to  overall  performance,  accounting  for 
15%  of  the  overall  performance  variance.  On  Course  2,  only  speed  control 
affected  overall  performance.  This  was  probably  due  to  the  high  proportion  of 
stagnated  moves  on  Course  2  as  compared  to  Course  1.  Care  must  be  used  in 
designing  discrete-time  tasks  to  ensure  that  there  is  sufficient  independence 
in  the  effects  of  the  decision  processes  to  permit  analysis  of  individual 
decision  components. 

IV.  CONCLUSIONS 


The  ability  to  investigate  individual  decision  process  can  be  use  to  gain 
greater  insight  into  the  cognitive  demands  of  vehicle  control  tasks.  Varying 
task  requirements,  vehicle  control  dynamics,  or  course  design,  may  make  it 


possible  to  change  the  relative  importance  of  differing  decision  components  in 
determining  overall  task  performance  or  the  difficulty  of  making  these 
decisions. 

An  additional  area  of  study,  which  has  not  been  addressed  in  the  present 
work,  is  decision  making  speed.  The  discrete-time  methodolgy  can  provide  a 
powerful  tool  for  studying  this  problem  by  permitting  the  unconfounding  of 
effects  arising  from  requirements  for  speeded  perception  and  speeded 
responding  from  those  due  to  the  decision  making  process. 
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